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1 .   INTRODUCTION 

This  report  considers  a  problem  of  selecting  from  a  set 
of  objects  a  subset  of  maximum  value  where  there  are  sequence 
dependent  constraints  which  restrict  the  choices  available. 
There  is  a  natural  ordering  among  the  objects  which  dictates 
the  order  in  which  they  must  be  considered.   It  is  assumed 
that  the  decision  maker  has  complete  knowledge  of  the  opportu- 
nities which  will  arise  so  that  the  problem  is  one  of  determining 
which  opportunities  to  select  and  which  to  forego  in  preparation 
for  later  gain.   The  problem  in  which  the  opportunities  have 
some  probability  of  vanishing  without  notice  is  considered 
briefly,  but  the  solution  method  is  the  same  as  long  as  these 
probabilities  are  known. 

The  problem  can  be  cast  in  several  forms.   Reference  1 
describes  the  problem  in  the  context  of  an  "investigator"  pass- 
ing through  a  region  in  which  there  is  a  number  of  points  to 
be  examined.   It  can  also  be  viewed  as  a  "delivery"  problem 
in  which  a  delivery  truck  has  a  list  of  locations  and  delivery 
times.   If  a  delivery  is  made,  it  must  be  done  at  the  given 
time  and  the  specified  location.   Knowing  the  transit  times 
from  point  to  point  and  the  value  of  each  customer,  the  problem 
is  to  select  the  most  valuable  subset  of  customers  to  serve. 

The  problem  can  be  viewed  as  a  job  shop  scheduling 
problem  in  which  the  production  manager  has  a  set  of  possible 
jobs  to  perform.   Each  job  has  a  required  starting  time  and 


known  set-up  and  processing  time.   The  problem  is  to  select 
the  most  valuable  subset  for  processing.   Reference  2  deals 
with  the  application  of  dynamic  programming  to  a  similar  problem 
in  which  each  job  has  an  availability  interval  within  which  it 
must  be  processed  if  selected. 

The  problem  is  viewed  here  as  one  in  which  a  defender  is 
guarding  some  region  against  attack  from  a  set  of  attackers 
whose  times  and  points  of  arrival  at  the  region  are  known.   The 
defender  is  constrained  to  remain  on  the  boundary  of  the  region 
and  must  move  to  the  point  of  intrusion  at  the  time  the  attacker 
arrives  in  order  to  destroy  the  attacker.   This  is  equivalent 
to  saying  that  the  defender  can  not  engage  the  target  until  it 
crosses  some  threshold  but  it  must  be  engaged  immediately  there- 
after.  The  selection  problem  arises  because  the  defender  is 

limited  in  the  rate  at  which  he  can  move  along  the  boundary. 

For  convenience  the  boundary  is  assumed  to  be  a  straight 

line  segment  along  which  the  defender  moves.   Other  configuration 
are  possible.  For  example,  some  problems  in  point  defense  can 
be  formulated  in  the  same  way.  In  that  case  the  boundary  can 
be  thought  of  as  a  circle  (of  zero  radius)  and  the  position  of 
the  defensive  system  is  represented  by  its  angular  rotation 
or  orientation.   Figure  1  illustrates  a  problem  of  this  type 
when  the  region  defined  is  the  portion  of  the  x-axis  from 
0  to   B. 
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FIGURE  1.   Illustration  of  Basic  Problem 


It  is  convenient  to  think  of  all  the  attackers  as 
having  the  same  speed  and  travelling  on  parallel  courses 
directly  toward  the  boundary,  but  it  is  not  necessary  to  do  so. 
In  fact,  all  that  is  required  is  the  time  and  point  of  arrival 
on  the  boundary  for  each  attacker.  It  is  assumed  hereafter 
that  the  attackers  are  numbered  in  order  of  arrival  at  the 
boundary  and  that  the  times  and  points  of  arrival  are  known. 
Let   t.   be  the  time  of  arrival  of  attacker   j   and   p. 
be  the  point  of  arrival  where   0    p.    B.   The  value  of 
attacker   j   is  represented  by   v..   It  is  also  convenient 
to  define  a  dummy  point  pn  ,  tn   with  value   vn  =  0   representing 
the  initial  location  of  the  defender. 

For  visual  presentation  of  the  problem  a  slight  trans- 
formation is  convenient.   This  problem  is  equivalent  to  one  in 
which  the  attackers  are  stationary  and  the  boundary  moves  in 
time  through  the  attackers.   As  an  example,  the  defender's 
position  in  the  two-dimensional  space   (x,t)   is  shown  in 
Figure  2. 

The  problem  remains  one  of  determining  the  optimal  set 
of  targets  for  the  defenders  to  engage  subject  to  limitations 
on  his  ability  to  reposition  his  defensive  system. 

This  report  deals  primarily  with  the  problem  in  which 
there  are  two  identical  defenders  although  the  ideas  are 
applicable  for  the  M-de fender  problem.   A  straightforward 
dynamic  programming  approach  for  the  M-defender  problem  is 
discussed  first.   See  also  Reference  1.   Next  an  efficient 
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FIGURE    2.       Space-Time    Representation 
of   Defender's    Position 


solution  method  for  the  problem  with  a  single  defender  is 
considered.   Then  an  improved  formulation  for  the  two-defender 
problem  is  presented  and  it  is  shown  how  that  problem  can  be 
solved  with  a  greatly  reduced  computational  burden.   A  computer 
problem  was  written  to  test  the  method  and  the  solution  to  a 
sample  problem  with  N  =  50  attackers  is  included. 

Several  generalizations  are  discussed  in  Section  5. 


2.   DYNAMIC  PROGRAMMING  SOLUTION  FOR  M-DEFENDERS 

The   M  investigator  problem  can  be  solved  with  a 
standard  dynamic  programming  approach  using  M  state  variables. 
Define  the  stages  such  that  stage   n   corresponds  to  the 
time   t  ,  and  let   X  =  (x,  ,  .  .  .  ,  x^  )   be  the  state  variable 
describing  the  location  along  the  boundary  of  each  investigator 
at  stage   n.   The  return  function  for  stage   n   is 


n   n     j 


x .   =  p   ,  any   i 
in    ^n      2 


r„(X_)  = 

0    otherwise. 


The  stage  transformation  functions  can  be  written  as 


n+i    n  n   n 


where   D    is  the  vector  of  decisions  prescribing  the  heading 
of  all  investigators  as  they  move  from  stage   n   to   n+1   and 

t    is  the  function  which  describes  the  resulting  locations 

n  3 


at  stage   n+l.   The  decisions   D    are  restricted  to  lie  in 

n 

some  set  K  .   An  alternative  formulation  simply  defines   5 

as  the  set  of  points  to  which  the  investigators  move  so  that 

jL,,  =  t  (x  ,D J  =  5  ,  but  still   5   is  constrained  to  lie 
n+l    n   n   n     n  n 

in  some  set  of  feasible  solutions,  say   K  . 

1         n 

The   complete   problem  can   be  written    as 


N 
maximize  T      r    (x    ,5    ) 

*-»  Tt  T\ 

n=0 


n     n      n 


sub 


ject    to         x    ,,    =    t    (x    ,D    ),         n   =    0,...,    N 
J  n+i  n      n      n 


Dn   c  Kn    ,  n  =  0 , .  . . ,    N 


and   can   be    solved   using   a   standard   dynamic   programming   approach 
with    tabular   computations    once   a   suitable   grid  has   been 
established   for    the    state    and   decision   variables.      The    solution 
requires    the   evaluation   of    the    following   recursive    equations 
beginning  with   stage      N 


fN+l(xN+l 


)    =    0 


where 


f    (x)    =      max        {r    (x.D)    +   f      ,  (x    , , )  }  /      n  =  0 , .  .  .  ,N , 

nn  _^Tr  nnn  n+l      n+i 

n     c  j\ 
n        n 


n+l  nnn 


This  method  is  very  general  and  can  be  used  to  solve 
several  variations  of  the  basic  problem,  except  that  it 
becomes  computationally  unwieldy  as   M   increases.   The  amount 
of  computation  required  rises  exponentially  with  M  but 
linearly  in   N. 

Two  generalizations  that  can  be  solved  using  a  similar 
formulation  are  the  problems  in  which  there  is  a  restriction 
for  each  defender  on  the  total  number  of  attackers  it  can 
engage  or  a  restriction  on  the  total  lateral  movement  of 
the  defender.   In  these  problems  the  vector  of  state  variables 
must  be  enlarged  to  include  a  component  for  the  number  of 
engagements  remaining  and  a  component  for  the  amount  of  lateral 
movement  remaining. 

The  formulation  above  is  not  practical  for  large 
values  of  M  nor  for  problems  with  constraints  which  lead 
to  additional  state  variables,  and  the  purpose  of  this  report 
is  to  present  a  method  which  is  more  efficient  than  that  just 
given,  but  first  an  efficient  method  for  the  basic  one-state- 
variable  problem  is  presented. 


3.   EFFICIENT  SOLUTION  FOR  ONE  DEFENDER 

For  the  problem  with  a  single  defender,  as  illustrated 

in  Figure  1,  the  algorithm  shown  in  Figure  3   provides  an 

efficient  solution  procedure.   That  algorithm  assumes  a  dummy 

point  at   p_   having  value   v„  =  0   representing  the  starting 

location  of  the  defender.   The  computation  produces  in  reverse 

order  the  sequence  of  optimal  return  functions   f  ,  n  =  0,...,N 

where  the  quantity   f    represents  the  optimal  total  return 

that  can  be  obtained  from  the  remaining  stages,  not  including 

n,  given  that  the  position  of  the  defender  is   d   at  time   t  . 

r  -  n  n 

The  sequence  of  optimal  decisions  is  also  produced  and  can  be 
used  beginning  at   d_   to  trace  the  optimal  policy.   The 
algorithm  relies  on  the  fact  that  the  only  relevant  value  of 
the  state  variable  at  any  stage  is  the  value  corresponding  to 
the  location  of  the  attacker  and  that  the  return  functions 
need  not  be  computed  for  any  other  values.   For  that  reason 

the  state  variable  is  suppressed  in   f  . 

L  r  n 

This    algorithm  can   easily  be    implemented   in   FORTRAN   as 
two   nested   DO   loops. 

4 .       IMPROVED    FORMULATION    FOR   THE    TWO-DEFENDER   PROBLEM 

This    section    shows   how   an    improved   formulation    for    the 
problem  with    two   defenders    can   be   obtained.       In    this    section   the 
N-attacker  problem   is    still   viewed  as    an      N      stage    dynamic 
programming  problem  where    stage      n      corresponds    to    the    time    tn 
at  which    the    defender   departs    from   the    target     n.      In    this 
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fn    =    0,    n    =    1,...,N 


Set      n   =   N 


n   =   n-1 


B  =  .17\J£si 


"*"  STOP 


|        No 


'>       No 


UNO 


i   =   n 


i   =   i+1 


I                \           Yes 
i   =   N+2?) »- 


No 


Is   it   feasible    for 

the    investigator   to   go 

from     p        to      p .  ? 


Yes 


F      -   v.    +    f. 
nxi 


F      >    f    ' 
k   n  n 


Yes 


Set      f      =   F 
n  n 

Set   decision      d      =    i 


FIGURE    3.      Algorithm   for    the   problem  with   a   single    defender 
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formulation,  as  in  the  formulation  of  Section  3,  passing  through 

stage  n  corresponds  to  engaging  target  n ;  and  if  target  n   is 

not  engaged,  then  stage   n   is  bypassed. 

At  stage  n  the  state  variables   x  ,  ,  x  _,   represent 

nl    nz     c 

the  positions  of  defenders  1  and  2  along  the  boundary.   For 

computations  it  is  assumed  that  a  grid  is  established  for  the 

state  variables.   For  convenience  this  grid  is  assumed  to  be  the 

same  at  each  stage,  consisting  of  S+l   points  equally  spaced  a 

distance   A  apart.   It  is  also  convenient  to  assume  that  the 

targets  occur  at  the  grid  points,  although  this  is  not  essential. 

If  they  were  located  between  grid  points  it  could  be  assumed  that 

a  target  is  engaged  if  the  defender  passes  within  a  distance   r 

of  the  target,  or  the  grid  points  could  be  redefined  at  each 

stage  to  include  the  target  location. 

One  observation  which  considerably  reduces  the  amount 

of  computation  is  that  if  the  defenders  pass  through  stage  n,  then 

one  of  the  defenders  must  be  located  at   p  ,  the  location  of  the 

n 

n   attacker.   Thus  instead  of   (S+l)    possible  values  of 

the  state  variables   x  ,   and   x  -   we  need  to  consider  only 

nl        nz 

(S+l)  +  (S+l)  -  1  =  2S  +  1   values.   The  computational  reduction 
from  this  observation  is  roughly  equivalent  to  reducing  the 
number  of  state  variables  by  one. 

A  second  observation  can  also  be  made.   This  is  that 
there  is  an  optimal  solution  in  which  the  two  defenders  do  not 
cross.   This  is  similar  to  the  observation  in  the  travelling 
salesman  problem  that  for  problems  with  Euclidean  distance 

11 


the   optimal    tour  need   not   cross    itself.      The   validity  of   this 

observation   can  be   established  by  noting   that   if   a   point  of 

crossing  occurs,    the   defenders    could   simply  be    renumbered 

so    that  number   1    remains    on    the   left   and  number    2   on    the    right. 

This    follows    from  the    fact   that   the   defenders   are    identical, 

and   it  would  not  hold   if   they  were   able    to   move    at   different 

rates   along  the   boundary.      It  would  also   be    invalid   if   a   penalty 

were    assessed   for  changing   the    direction   of  motion   from  left 

to   right  or   vice-versa  or   if   they  had  constraints   on   the   number  of 

attackers  handled. 

With  this  second  observation  the  number  of  possible 
values  of  the  state  variables  is  reduced  even  further.  The 
possible  values  of  the  state  variables  at  stage   n,  x  ,   and 


x 


~ ,  are  given  in  Table  1  where   p    is  the  x-coordinate  of 
n2      3  *n 

the  n    target.   Since  the  targets  were  initially  assumed  to 
lie  on  the  grid  points   p   =  KA   for  some   K.   The  state  variable 
combinations  listed  in  Table  1  have  been  numbered  for  reference, 
but  notice  that  knowing  the  state  number  is  equivalent  to 
having  the  value  of  both  state  variables.   For  example,  state 
number   K1   implies  that 


A  ,   if    K'A  <  p. 


K' A  >  p. 


and 


x  n 
n2 


(  p±A  ,   if    K'A  <  p± 


K'A  ,        K' A  >  p. 
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State 

Number 

nl 

Xn2 

0 

o- 

Pn 

1 

• 

A 

Pn 

• 

K-l 

(K-l)A 

Pn 

K 

Pn 

(K+l) A 

• 

S 

Pn 

• 
• 

(S+l) A 

TABLE  1 

Possible  values  of  the  state  variable 
in  the  two  defender  problem. 
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Thus  all  the  state  variable  information  is  conveyed  by  the 

state  numbers  and  it  is  unnecessary  to  write  both  state 

variables.   For  this  reason  it  is  adequate  to  let  the  single 

state  variable  at  stage  n  be  the  state  number  S    so  that 

3  n 

S      =    0,1,2,...,    S.      The    result  of   this    second  observation   is    to 
n 

reduce    the   number  of   state   variable   combinations   at  each    stage 
from  2S+1    to   S+l. 

In    the   dynamic   programming   formulation      f    (S    )    denotes 
the   maximum  total    return   from   the   remaining  stages      n+l,...,N 
given   that   the    defenders    are    at   stage   n    in    state      S    .      The 
general    recursion   can  be  written    as 


w  =  ° 


f    (S    )    =      max  {v.      +    f.    (D    )}     , 

n     n           -      _  o             5        n 

<5    ,  D  n             n 
n      n 


5       >n,    D      6k       (S    )       n=l  m-i 


n 

where   5    is  a  decision  variable  denoting  the  stage  to  which 
n 

the  defender  moves  next  and  D    is  the  decision  variable  which 

n 

determines  the  state  number  to  which  each  defender  moves  at 

stage   6  .   The  set  of  feasible  states  at  stage   5    given 

state   S    at  stage  n  is  denoted  by   K.  (S  ) . 
n        3  J    0    n 

n 

From  state   S    it  may  not  be  possible  to  go  to  some 

stages  and,  among  those  that  are  possible,  the  question  of  which  to 

enter  must  be  resolved.   The  general  computation  proceeds  as 

shown  in  Figure  4  beginning  at  stage  N-l.   Figure  4  assumes 

completion  of  the  computation  of   f  (S  )   for  n  =  N-l,  N-2,  .-.  .  ,  i+. 

so  that  the  next  step  is  to  evaluate   f.(-). 

l 
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Go  To 

Next  Stage 

i  -  1 


YES 


YES 


NO 


NO 


Consider  Stage  i 
Si  =  -A,     f(Si)  =  0 

6.=  i 

S.  =  -A 
5i 


S .  =  S.  +  A 
X      1 


Si  =  (S+1)A? 


NO 


5i  =  6i  +  l 


-«— <S.     =  N  +  1  ? 


No 


=  Sr   +  A 


s5   =  (S+1)A  ? 
i      


NO 


Is  It  Feasible 
To  Go  From 

S.   to   Sx  ? 

l       5  • 

l 


YES 


Is  It  Desirable 
To  Do  So? 


i.e.  is 


v5.  +  f«.(S«.) 


the  largest  value 
considered  so  far? 


/"\ 


YES 


YES 


Save 
5  .  and  S - 

1         6 


FIGURE  4 
15 


5.   MODIFICATIONS  TO  THE  BASIC  PROBLEM 

A  similar  problem  in  which  each  target   j   has  some 
probability   1  -  p.   of  vanishing  before  it  is  engaged  can 
be  solved  by  the  same  method.   The  objective  becomes  maximization 
of  expected  value  and  the  computation  is  modified  by  diminishing 
the  value  of  target   j   from  v.   to   p. v..   After  this  change, 
the  remainder  of  the  solution  method  is  the  same. 

Several  other  modifications  of  the  basic  problem  are  of 
interest.   These  include  the  problem  in  which  there  is  a 
limitation  on  the  total  motion  of  the  defender  as  would  be  the 
case  where  fuel  or  some  other  resource  is  consumed.   Another 
interesting  problem  arises  when  there  is  a  constraint  on  the 
total  number  of  attackers  which  can  be  engaged  by  each  defender. 
Both  of  these  modifications  can  be  handled  in  the  basic  dynamic 
programming  formulation  by  the  addition  of  another  state  vari- 
able; but  as  mentioned  previously  large  problems  require  that 
more  efficient  methods  be  found. 

In  the  computer  program  which  implements  this  solution 
method  for  the  two-defender  problem  a  further  savings  is 
incorporated.   If  the  difference  between   6 .   and   i   is 
large,  or  at  least  if  the  time  difference  between  the  stages  is 
large,  it  is  almost  certainly  possible  to  engage  some  target 
between   i   and   5 . ,  say  at  stage   j ,  and  still  move  to  any 
desired  state  at  stage   5  . .   In  such  a  case  it  is  unnecessary 
for  the  recursive  procedure  to  examine  stacres   5.,  5  .  ,,,...,  N 

sr  3       1     1  +  1 

as  possible  choices  of  the  next  stage  following   i.   This  is 
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because  those  choices  are  dominated  by  another  choice,  namely, 
moving  to  stage  j  from  i.   A  test  to  this  effect  was  incorporated 
with  a  considerable  reduction  in  computer  time.   The  usefulness 
of  such  a  test  depends  on  the  particular  rules  of  motion  assumed 
for  the  defender  and  will  be  pursued  here. 

Reference  3  considers  the  problem  from  the  point  of  view 
of  two  attackers  moving  through  a  region  containing   N   targets . 
Each  attacker  must  select  a  subset  of  the  targets  and  no  target 
can  be  selected  by  both  attackers .   The  case  in  which  the 
attackers  enter  the  region  from  different  directions  is  considered 
That  case  is  not  covered  here  since  the  ordering  of  the  stages 
differs  for  the  two  attackers. 


6 .   COMPUTER  PROGRAM 

A  program  was  written  in  FORTRAN  IV  to  implement  the 
procedure  described  in  Section  4  for  the  two  defender  problem. 
This  program  was  used  to  solve  several  problems  in  which  the 
attacker's  locations  were  generated  randomly  using  a  uniform 
distribution  to  determine  the  coordinates   x   and  y,  where 
0  <_  x  <  11  and   0  <_  t    100.   In  order  to  make  the  x-coordinates 
fall  on  a  conveniently  spaced  grid,  the  x-values  generated  were 
truncated  to  an  integer  before  solving  the  problem. 

Table  2  shows  the   x   and   t   coordinates  for  a  sample 
problem  involving  50  attackers.   Also  shown  are  the  randomly 
generated  target  values  ranging  from  1  to  5 . 
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A  graphical  presentation  of  the  sample  problem  is 
given  in  Figure  5.   The  numbers  beside  each  point  are  the 
target  values  on  the  right  and  the  point  number  (mod  10)  on 
the  left.   The  two  lines  show  the  sets  of  attackers  engaged 
by  the  two  defenders.   The  roles  of  motion  used  in  this  sample 
problem  permitted  the  defenders  to  move  left  or  right  at  the 
rate  of  one  unit  per  unit  time .   The  scale  used  in  plotting 
Figure  5  is  compressed  in  the  t  direction  and  it  gives  the 
appearance  that  the  defenders  can  move  more  rapidly  than  one 
unit  per  unit  time,  but  reference  to  the  coordinates  of  each 
point  will  confirm  that  the  rules  of  motion  are  not  violated. 
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1 

5.00 

0.10 

1 

2 

6  .00 

2.33 

4 

3 

6.00 

5.20 

1 

4 

4.00 

5  .53 

2 

5 

7.00 

7.74 

2 

6 

6.00 

8.96 

3 

7 

6.00 

9  .31 

1 

8 

4.00 

16.18 

3 

9 

3.00 

17.18 

4 

10 

4.00 

20.88 

4 

11 

2.00 

20.80 

5 

12 

6.00 

21.37 

1 

13 

4.00 

22.19 

4 

14 

9.00 

22.81 

5 

15 

1.00 

26.1'6 

2 

16 

9.00 

26.91 

3 

17 

6  .00 

31.96 

1 

18 

1.00 

32.69 

5 

19 

7.00 

42.18 

4 

20 

7.00 

43.04 

3 

21 

3.00 

44.45 

2 

22 

4.00 

45.69 

2 

23 

5.00 

49.25 

5 

24 

7.00 

50.33 

3 

25 

8.00 

53.22 

5 

26 

2.00 

55.22 

2 

27 

7.00 

59.42 

1 

28 

3.00 

59.88 

5 

29 

2.00 

60.38 

3 

30 

9.00 

60.63 

4 

31 

8.00 

65.35 

2 

32 

6.00 

65.54 

5 

33 

8.00 

66.17 

3 

34 

3.00 

65  .79 

2 

35 

3.00 

66.89 

2 

36 

2.00 

69.05 

2 

37 

9.00 

70.86 

3 

38 

4.00 

72.45 

5 

39 

0.00 

73.54 

4 

40 

8.00 

75.19 

3 

41 

6.00 

76.55 

3 

42 

4.00 

76.92 

2 

43 

7.00 

80  .05 

2 

44 

0  .00 

81.85 

2 

45 

4.00 

84.41 

1 

46 

9.00 

85.02 

3 

47 

6  .00 

90.00 

1 

48 

7.00 

90.34 

5 

49 

8.00 

92.90 

1 

50 

4.00 

99.97 

5 

Table  2.   Data  for  Sample  Problem  Involving  50  Attackers 
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